What is claimed is: 

1. A data multiplexing method comprising the 
steps of: 

storing a video stream and an audio stream in 
5 first and second buffers, respectively; 

comparing said video stream and audio stream from 
said first and second buffers with each other to 
determine an order of packets and a parameter in such a 
way that in generation of a multiplexed stream to be 
10 outputted, a difference of playback start time or a 

difference of playback end time of video and audio to 
be recorded in a same video object unit becomes equal 
to or smaller than a threshold; and 

dividing said video stream and audio stream from 
15 said first and second buffers into data blocks in 

accordance with said parameter and adding headers to 
said data blocks to produce packets, multiplexing said 
packets and outputting a multiplexed stream. 

20 2. A data recorded medium on which a multiplexed 

stream is recorded that includes a video stream and an 
audio stream and is acquired by dividing each of said 
video stream and said audio stream into data blocks, 
adding headers to said data blocks to produce packets 

25 and multiplexing said packets, 

said multiplexed stream being such that a 
difference of playback start time or a difference of 
playback end time of video and audio included in a 



specific one of video ob j ect units each having a series 
of packets including at least one group of pictures, 
which is a video encoding unit in said video stream, 
becomes egual to or smaller than a threshold. 

5 

3. The data recorded medium according to claim 1, 
wherein said threshold is an integer multiple of a 
duration time of reproduction of an audio access unit. 

10 4. A data recorded medium on which a multiplexed 

stream is recorded that includes a video stream and an 
audio stream and is acquired by dividing each of said 
video stream and said audio stream into data blocks, 
adding headers to said data blocks to produce packets 

15 and multiplexing said packets and which is of a model 
that allows said multiplexed stream to be read in a 
buffer of a decoder packet by packet and to be 
outputted picture by picture and audio access unit by 
audio access unit in accordance with a playback time, 

20 said multiplexed stream being such as to satisfy 

a condition of Tea < Tsv where Tsv is a read start time 
of a head packet in a specific one of video object 
units each having a series of packets including at 
least one group of pictures, which is a video encoding 

25 unit in said video stream, and Tea is a read end time 
of a packet including data of a last portion of an 
audio access unit having a playback end time after but 
closest to a playback start time of video included in 



said specific video object unit or an audio access unit 
having a playback end time after but closest to that 
playback start time. 

5 5. The data recorded medium according to claim 4, 

wherein said threshold is an integer multiple of a 
duration time of reproduction of said audio access unit. 

6 . A data recording apparatus for dividing each 
10 of a plurality of streams including a video stream and 
an audio stream into data blocks, adding headers to 
said data blocks to produce packets, multiplexing said 
packets and recording said multiplexed packets, 
comprising : 

15 first buffer means for storing said video stream; 

second buffer means for storing said audio 
stream ; 

a control section for comparing said video stream 
from said first buffer means with said audio stream 

20 from said second buffer means to determine an order of 
packets and a parameter in such a way that a difference 
of playback start time or a difference of playback end 
time of video and audio to be recorded in a same video 
object unit in a multiplexed stream to be outputted 

25 becomes equal to or smaller than a threshold and 
outputting said parameter; and 

a multiplexing section for dividing said video 
stream from said first buffer means and said audio 
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stream from said second buffer means into data blocks 
in accordance with said parameter and adding headers to 
said data blocks to produce packets, multiplexing said 
packets and outputting a multiplexed stream. 

5 

7. The data recording apparatus according to 
claim 6, wherein said threshold is an integer multiple 
of a duration time of reproduction of an audio access 
unit. 

10 

8 . A program for allowing a computer to execute 
procedures of dividing each of a plurality of streams 
including a video stream and an audio stream into data 
blocks, adding headers^ to said data blocks to produce 

15 packets, multiplexing said packets and recording said 
multiplexed packets, comprising: 

a first step of storing said video stream and 
said audio stream in buffers; 

a second step of comparing said video stream and 
20 audio stream read from said buffers with each other to 
determine a multiplexing timing in such a way that a 
difference of playback start time or a difference of 
playback end time of video and audio to be recorded in 
a same video object unit in a multiplexed stream to be 
25 outputted becomes equal to or smaller than a threshold 
and outputting said parameter; and 

a third step of multiplexing video packets and 
audio packets in accordance with said multiplexing 
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timing and outputting a multiplexed stream. 



9. The program according to claim 8, wherein 
said threshold is an integer multiple of a duration 
5 time of reproduction of an audio access unit. 



10. The program according to claim 8, wherein 
said second step includes : 

a fourth step of discriminating and separating 
10 said video stream and said audio stream; 

a fifth step of detecting a picture in said video 
stream and calculating a playback time; 

a sixth step of detecting an audio access unit of 
said audio stream and calculating a playback time; 
15 a seventh step of determining a data division 

position and making data blocks for said video stream; 

an eighth step of determining a data division 
position and making data blocks for said audio stream; 

a ninth step of calculating a capacity of a 
20 virtual video buffer for said video stream and 

calculating a range of a video multiplexing timing; 

a tenth step of calculating a capacity of a 
virtual audio buffer for said audio stream and 
calculating a range of an audio multiplexing timing; 
25 an eleventh step of determining final 

multiplexing timings for video and audio packets from 
said playback times and said capacities of said virtual 
buffers for said video stream and said audio stream; 
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and 

a twelfth step of adding headers to said data 
blocks to produce packets and outputting said packets 
in accordance with said final multiplexing timings. 

5 

11. The program according to claim 10, wherein 
said threshold is an integer multiple of a duration 
time of reproduction of said audio access unit. 

10 12. A decoder comprising: 

a demultiplexer for separating at least a video 
stream and an audio stream from a multiplexed stream 
including said video stream and said audio stream; 

a video decoding section for playing back video 
15 via a video buffer for buffering said separated video 
stream; and 

an audio decoding section for playing back audio 
via an audio buffer for buffering said separated audio 
stream, 

20 a capacity of said audio buffer being set equal 

to or larger than a value obtained by adding a maximum 
value of a delay time of a picture in said video stream 
in said multiplexed stream with respect to a video 
buffering verifier and a playback time of one frame of 

25 said video stream together and multiplying a resultant 
value by a maximum bit rate of said audio stream. 
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